21 research outputs found
Benchmarking projective simulation in navigation problems
Projective simulation (PS) is a model for intelligent agents with a
deliberation capacity that is based on episodic memory. The model has been
shown to provide a flexible framework for constructing reinforcement-learning
agents, and it allows for quantum mechanical generalization, which leads to a
speed-up in deliberation time. PS agents have been applied successfully in the
context of complex skill learning in robotics, and in the design of
state-of-the-art quantum experiments. In this paper, we study the performance
of projective simulation in two benchmarking problems in navigation, namely the
grid world and the mountain car problem. The performance of PS is compared to
standard tabular reinforcement learning approaches, Q-learning and SARSA. Our
comparison demonstrates that the performance of PS and standard learning
approaches are qualitatively and quantitatively similar, while it is much
easier to choose optimal model parameters in case of projective simulation,
with a reduced computational effort of one to two orders of magnitude. Our
results show that the projective simulation model stands out for its simplicity
in terms of the number of model parameters, which makes it simple to set up the
learning agent in unknown task environments.Comment: 8 pages, 10 figure
Projective simulation with generalization
The ability to generalize is an important feature of any intelligent agent.
Not only because it may allow the agent to cope with large amounts of data, but
also because in some environments, an agent with no generalization capabilities
cannot learn. In this work we outline several criteria for generalization, and
present a dynamic and autonomous machinery that enables projective simulation
agents to meaningfully generalize. Projective simulation, a novel, physical
approach to artificial intelligence, was recently shown to perform well in
standard reinforcement learning problems, with applications in advanced
robotics as well as quantum experiments. Both the basic projective simulation
model and the presented generalization machinery are based on very simple
principles. This allows us to provide a full analytical analysis of the agent's
performance and to illustrate the benefit the agent gains by generalizing.
Specifically, we show that already in basic (but extreme) environments,
learning without generalization may be impossible, and demonstrate how the
presented generalization machinery enables the projective simulation agent to
learn.Comment: 14 pages, 9 figure
A Qubit-Efficient Variational Selected Configuration-Interaction Method
Finding the ground-state energy of molecules is an important and challenging
computational problem for which quantum computing can potentially find
efficient solutions. The variational quantum eigensolver (VQE) is a quantum
algorithm that tackles the molecular groundstate problem and is regarded as one
of the flagships of quantum computing. Yet, to date, only very small molecules
were computed via VQE, due to high noise levels in current quantum devices.
Here we present an alternative variational quantum scheme that requires
significantly less qubits. The reduction in qubit number allows for shallower
circuits to be sufficient, rendering the method more resistant to noise. The
proposed algorithm, termed variational quantum
selected-configuration-interaction (VQ-SCI), is based on: (a) representing the
target groundstate as a superposition of Slater determinant configurations,
encoded directly upon the quantum computational basis states; and (b) selecting
a-priory only the most dominant configurations. This is demonstrated through a
set of groundstate calculations of the H, LiH, BeH, HO, NH and
CH molecules in the sto-3g basis set, performed on IBM quantum devices.
We show that the VQ-SCI reaches the full-CI (FCI) energy within chemical
accuracy using the lowest number of qubits reported to date. Moreover, when the
SCI matrix is generated ``on the fly", the VQ-SCI requires exponentially less
memory than classical SCI methods. This offers a potential remedy to a severe
memory bottleneck problem in classical SCI calculations. Finally, the proposed
scheme is general and can be straightforwardly applied for finding the
groundstate of any Hermitian matrix, outside the chemical context.Comment: 32 pages, 5 figure
Quantum speedup for active learning agents
Can quantum mechanics help us in building intelligent robots and agents? One
of the defining characteristics of intelligent behavior is the capacity to
learn from experience. However, a major bottleneck for agents to learn in any
real-life situation is the size and complexity of the corresponding task
environment. Owing to, e.g., a large space of possible strategies, learning is
typically slow. Even for a moderate task environment, it may simply take too
long to rationally respond to a given situation. If the environment is
impatient, allowing only a certain time for a response, an agent may then be
unable to cope with the situation and to learn at all. Here we show that
quantum physics can help and provide a significant speed-up for active learning
as a genuine problem of artificial intelligence. We introduce a large class of
quantum learning agents for which we show a quadratic boost in their active
learning efficiency over their classical analogues. This result will be
particularly relevant for applications involving complex task environments.Comment: Minor updates, 14 pages, 3 figure